Query by Committee

We propose an algorithm called query by com mittee in which a committee of students is trained on the same data set The next query is chosen according to the principle of maximal disagreement The algorithm is studied for two toy models the high low game and perceptron learning of another perceptron As the number of queries goes to in nity the committee algo rithm yields asymptotically nite information gain This leads to generalization error that decreases exponentially with the number of ex amples This in marked contrast to learning from randomly chosen inputs for which the in formation gain approaches zero and the gener alization error decreases with a relatively slow inverse power law We suggest that asymptot ically nite information gain may be an impor tant characteristic of good query algorithms